InstanceFormer: An Online Video Instance Segmentation Framework

Recent transformer-based offline video instance segmentation (VIS) approaches achieve encouraging results and significantly outperform online approaches. However, their reliance on the whole immense computational complexity caused by full Spatio-temporal attention limit them in real-life applications such as processing lengthy videos. In this paper, we propose a single-stage efficient VIS framework named InstanceFormer, which is especially suitable for long challenging We three novel components to model short-term long-term dependency temporal coherence. First, propagate representation, location, semantic information of prior instances changes. Second, memory cross-attention decoder, allows network look into earlier within certain window. Finally, employ contrastive loss impose coherence representation an across all frames. Memory are particularly beneficial long-range modeling, including scenarios like occlusion. The proposed InstanceFormer outperforms previous benchmark methods large margin multiple datasets. Most importantly, surpasses datasets YouTube-VIS-2021 OVIS. Code available at https://github.com/rajatkoner08/InstanceFormer.